Audio-visual data capturing has been implemented in some command and control systems, such as operating room control systems. For example, devices such as the Karl Storz Advanced Image and Data Archiving System (AIDA™) allow a surgeon to capture a video stream signal or still images obtained during a surgical procedure. The image or video recording may further be annotated by a user, such as a surgeon performing an operation. Generally the voice annotations are recorded and the recordings are stored with the captured still or video image. The captured audio and visual data is generally stored in a database or on portable media (e.g., Compact Disk).
In the medical field, the captured audio and visual data may include important information related to the patient and/or the treatment being given to the patient. For example, the captured audio and visual data may be used post-surgery for obtaining additional medical information about a particular patient and for making treatment decisions. The audio and visual data may also be used for training purposes and for documenting medical procedures for potential liability causes of action. However, prior art audio and visual data capturing systems have limitations.
One disadvantage of prior art audio and visual capturing systems is that only raw audio-visual data, i.e., visual imagery and voice recordings, is captured and stored. Therefore, the usability of the data is very limited. In order to find any desired data, e.g., related to particular subject, a user must play the recording and listen and/or watch for all instances of the desired information. This is both a cost ineffective and inaccurate means to obtain the desired data. It is therefore desired to provide an improved audio and visual capturing system providing highly useable data.
A further disadvantage of prior art systems is that device status information is not captured, synchronized, and stored along with any pertinent or desired audio-visual data. Examples where this would be highly desired is where during an operation or procedure, a medical professional wishes to document a particular event; such as abnormal bleeding or other such complication. Along with the audio data being captured, transcribed, and synchronized with visual data, patient vital signs, such as those monitored by associated medical device(s), may also be captured and synchronized. Moreover, medical device operational status data, such as pump and vacuum pressures, medication levels being received by the patient, anesthesia device settings, and the like, may be captured and synchronized with the audio-visual data. Similarly, technical and scientific endeavors would also benefit from a real-time capture and synchronization of audio, visual, and device data during research and development, testing, and/or system monitoring endeavors. It is therefore desired to provide a system for capturing audio, visual and device data.
Another disadvantage of prior art systems is that dynamic or simultaneous control of equipment via a speech recognition system is difficult to achieve with add-on audio transcription sub-systems. This difficulty arises when user audio utterances, which are intended by the system user as commands to control equipment, are misinterpreted as audio data intended to be captured, synchronized, transcribed, and stored on a real-time basis. Furthermore, add-on audio transcription sub-systems lack sufficient ability to learn and adapt to new topics of conversation. In a typical system or application, language or topic changes can happen very frequently. Changes may occur slowly over time or abruptly, such as when a device is added to or removed from the system. Prior art systems are unable to properly adapt to these changes and therefore have a high initial transcription error rate.
What is highly desired then is a speech recognition command and control system, which discriminates between user audio utterances intended for equipment command and control from audio data intended for capture, synchronization transcription, and storage with other data.